Message Format
The Message Format tab will only be shown if the task is defined with a streaming target endpoint that supports this feature.
When a task is defined with such an endpoint, you can specify a custom message format that will override the default Replicate message format. This may be useful if the consumer application needs to process the message in a particular format.
The custom message format can be defined at task level and/or at table level. When it is defined at both task and table level, the message format defined for the table will take precedence over the message format defined for the task.
For information on defining a custom message format at table level, see Message format.
-
Supported by the Kafka target endpoint only
-
The custom message format feature cannot be used in conjunction the Attunity Avro Message Decoder SDK
-
Hierarchical structured messages are not supported.
General rules and usage guidelines
When defining a custom message, it is import to consider the rules and usage guidelines listed below.
Section names:
The following naming rules apply:
- Section names must start with the characters a-z, A-Z, or _ (an underscore) and can then be followed by any of the following characters: a-z, A-Z, 0-9, _
- With the exception of the Record name and Key name sections (that do not end with a slash), removing the slash from section names will flatten the hierarchy of the associated section (see Slashes below).
- All section names except Record name and Key name can be deleted (see Deletion: below)
-
The Data name and Include record Before-data section names cannot both be deleted
-
The Data name and Include record Before-data section names cannot be the same
Some of the section names in the UI end with a slash (e.g. beforeData/). The purpose of the slash is to maintain a hierarchy of the different sections within the message. If the slash is removed, the following will occur:
- The hierarchical structure of that section will be flattened, resulting in the section name being removed from the message
- The section name will be prefixed to the actual metadata, either directly or using a separator character (e.g. an underscore) that you appended to the name
Example of a data message when headers/ is specified with a slash:
"message":{ "data":{ "COL1": "159", "COL2": "159" }, "beforeData": null, "headers": { "operation": "INSERT", "changeSequence": "2018100811484900000000233",
Example of a data message when headers_ is specified with an underscore instead of a slash:
"message":{ "data":{ "COL1": "159", "COL2": "159" }, "beforeData": null, "headers_operation": "INSERT", "headers_changeSequence": "2018100811484900000000233",
Deleting a section name from the message will flatten the hierarchical structure of that section. This will result in all of that section's data appearing immediately below the content of the preceding section.
Example of a data message with the headers section name:
"message":{ "data":{ "COL1": "159", "COL2": "159" }, "headers": { "operation": "INSERT", "changeSequence": "2018100811484900000000233",
Example of a data message without the headers section name:
"message":{ "data":{ "COL1": "159", "COL2": "159" }, "beforeData": null, "operation": "INSERT", "changeSequence": "2018100811484900000000233",
Variables
You can add variables to names by clicking the button at the end of the row. The following variables are available:
- SERVER_NAME - The host name of the Replicate Server
- TARGET_TABLE_NAME - The name of the table
- TARGET_TABLE_OWNER - The table owner
- TASK_NAME - The name of the Replicate task
The TARGET_TABLE_OWNER variable is not available for the Record name and Key name options (described in the table below).
Defining a custom message format
To define a custom message format, click the Custom Message Format button and configure the options as described in the table below.
To revert to the default message format, click the Default Message Format button.
Option | Description |
---|---|
A unique identifier for the message. This should be a string, separated by periods. Note that the Namespace will be included in both the message and the message key. Example: mycompany.queue.msg Default name: com.attunity.queue.msg.{{TASK_NAME}}.{{TARGET_TABLE_OWNER}}.{{TARGET_TABLE_NAME}} Default name for Control Tables: com.attunity.queue.msg.{{TARGET_TABLE_NAME}} Information note
The default Control Table Namespace cannot be changed via the UI. You can, however, change the default Control Table Namespace as follows:
For instructions on exporting/importing tasks, see Migrating tasks. |
|
Record name |
The name of the record (i.e. message). Default name: DataRecord |
Data name |
All of the data columns included in the record. Default name: data/ |
Include Headers |
Header columns provide additional information about the source operations. For more information, see Include the following headers below. |
Include headers namespace |
A unique identifier for the header columns section. This should be a string, separated by periods. Example: headers.queue.msg Default name: com.attunity.queue.msg |
Headers name |
The name of the section containing the Replicate column headers. Default name: headers/ |
For a description of the available header columns, see Data Messages in Metadata and data messages. Default: All columns are included by default, except the External Schema ID column. |
|
Include record Before-data |
When this check box is selected (the default), both pre and post UPDATE data will be included in UPDATE messages. To include only the post UPDATE data in messages, clear the check box. Default name: beforeData/ |
Key name |
The name of the section containing the message key. Default name: keyRecord This option is only applicable when:
|